LIPSCHITZ BEHAVIOR OF THE ROBUST REGULARIZATION 



Abstract. To minimize or upper-bound the value of a function "ro- 
bustly", we might instead minimize or upper-bound the "e-robust regu- 
larization", defined as the map from a point to the maximum value of the 
function within an e-radius. This regularization may be easy to compute: 
convex quadratics lead to semidefinite-representable regularizations, for 
example, and the spectral radius of a matrix leads to pseudospectral com- 
putations. For favorable classes of functions, we show that the robust 
regularization is Lipschitz around any given point, for all small e > 0, 
even if the original function is nonlipschitz (like the spectral radius). One 
such favorable class consists of the semi-algebraic functions. Such func- 
tions have graphs that are finite unions of sets defined by finitely-many 
polynomial inequalities, and are commonly encountered in applications. 
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1. Introduction 

In the implementation of the optimal solution of an optimization model, 
one is not only concerned with the minimizer of the optimization model, 
but how numerical errors and perturbations in the problem description and 
implementation can affect the solution. We might therefore try to solve 
an optimization model in a robust manner. The issues of robust optimiza- 
tion, particularly in the case of linear and quadratic programming, are doc- 
umented in [HI. 

A formal way to address robustness is to consider the "robust regular- 
ization" [|T5l . The notation denotes a set- valued map. That is, if 
F : X and x e X, then F (x) is a subset of Y. 

Definition 1.1. For e > and F : X ^ R™, where X C R", the set-valued 
robust regularization F^ : X ^ is defined as 

F, (x) := {F (x + e) I |e| < e, a; + e G X} . 

For the particular case of a real- valued function f : X ^ M, we define the 
robust regularization : X ^ R of / by 

f,{x) := sup{yef,{x)} 

= sup {y I 3x' G X such that / (x) = y and \x' — x\ < e} . 

In this paper, we restrict our attention to the real-valued robust regular- 
ization : X — R. The use of set-valued analysis is restricted to Section 

m 

The minimizer of the robust regularization protects against small pertur- 
bations better, and might be a better solution to implement. We illustrate 
with the example 




—X if X < 
^/x if X > 0. 



The robust regularization can be quickly calculated to be 

^ , . _ J e — X if X < a (e) 
^'^^^ ^ \ ^fTT^ ifx>a(e), 

where a (e) = ^+2':-Vi+8 e ^ _^ jj^g minimizer of / is a (0), and / is not 
Lipschitz there. To see this, observe that ^^^^^J^^^ ^ oo as 5 ^ 0. But the 
robust regularization is Lipschitz at its minimizer a (e) ; its left and right 
derivatives there are —1 and — , ^ , which are both finite. 

The sensitivity of / at can be attributed to the lack of Lipschitz conti- 
nuity there. Lipschitz continuity is important in variational analysis, and is 
well studied in the recent books |l23l|20l|. The existence of a finite Lipschitz 
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constant on / close to the optimizer can be important in the problems from 
which the optimization problem was derived. 

There are two main aims in this paper. The first aim is to show that robust 
regularization has a regularizing property: Even if the original function / is 
not Lipschitz at a point x, the robust regularization can be Lipschitz there 
under various conditions. For example, in Corollary 14.61 we prove that if 
the set of points at which / is not Lipschitz is isolated, then the robust regu- 
larization is Lipschitz at these points for all small e > 0. The second aim 
is to highlight the relationship between calmness and Lipschitz continuity, 
a topic important in the study of metric regularity, and studied in some gen- 
erality for set-valued mappings (for example, in IfTTl Theorem 2.1], [|2T1 
Theorem 1.5]) but exploited less for single- valued mappings. 

In Theorem l5.3[ we prove that if / : ^ M is semi-algebraic and con- 
tinuous, then given any point in M", the robust regularization is Lipschitz 
there for all small e > 0. Semi-algebraic functions are functions whose 
graph can be defined by a finite union of sets defined by finitely many poly- 
nomial equalities and inequalities, and is a broad class of functions in appli- 
cations. (For example, piecewise polynomial functions, rational functions 
and the mapping from a matrix to its eigenvalues are all semi-algebraic 
functions.) Moreover, the Lipschitz modulus of at x is of order o (i). 
This estimate of the Lipschitz modulus can be helpful for robust design. 

Several interesting examples of robust regularization are tractable to com- 
pute and optimize. For example, the robust regularization of any strictly 
convex quadratic is a semidefinite -representable function, tractable via 
semidefinite programming: see Section [6l The robust regularizations of the 
spectral abscissa and radius of a nonsymmetric square matrix, which are 
the largest real part and the largest norm respectively of the eigenvalues of a 
matrix, are two more interesting examples. The robust regularization of the 
spectral abscissa and spectral radius are also known as the pseudospectral 
abscissa and the pseudospectral radius. The pseudospectral abscissa is im- 
portant in the study of the system (t) = Au {t), and is easily calculated 
using the algorithm in [4], while the pseudospectral radius is important in 
the study of the system Ut+i = Aut, and is easily calculated using the algo- 
rithm in IfTSl . We refer the reader to [|27l for more details on the importance 
of the pseudospectral abscissa and radius in applications. The spectral ab- 
scissa is nonlipschitz whenever the eigenvalue with the largest real part has 
a nontrivial Jordan block. But for a fixed matrix, the pseudospectral ab- 
scissa is Lipschitz there for all e G (0, e) if e > is small enough [16]. We 
rederive this result here, using a much more general approach. 
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2. Calmness as an extension to Lipschitzness 

We begin by discussing the relation between calmness and Lipschitz con- 
tinuity, which will be important in the proofs in Section [5] later. Throughout 
the paper, we will limit ourselves to the single-valued case. For more on 
these topics and their set- valued extensions, we refer the reader to [|23l . 

Definition 2.1. Let F : X ^ M" be a single-valued map, where X c M". 

(a) [[231 Section 8F] Define the calmness modulus of F at x with respect 
to X to be 

calmF (x) := inf {k | There is a neighbourhood V of x such that 

\F (x) - F {x)\ < k\x - x\ for all x e V n X} 

\Fix)-F{x)\ 
— limsup j — . 

X — >x \X X\ 

X 

Here, x — ^ x means that x E X and x ^ x. The function F is calm at x 

X 

with respect to X if calm F (x) < oo. 

(b') [|23l Definition 9.1] Define the Lipschitz modulus of F at x with re- 
spect to X to be 

lip F (x) := inf {k | There is a neighbourhood V of x such that 

\F{x) -F{x')\ < k\x-x \ forallx,x' e V n X} 



= lim sup 



\F{x) - 


F{x')\ 


\x — 


x'\ 



The function F is Lipschitz at x with respect to X if lip F (x) < oo. o 

The definitions differ slightly from that of [|23l[ . As can be seen in the 
definitions, Lipschitz continuity is a more stringent form of continuity than 
calmness. In fact, they are related in the following manner. 

Proposition 2.2. Suppose that F : X where X C M". 

(a) limsup^ ^^calmF(x) <lipF(x). 

X 

(b) If there is an open set U containing x such that U (IX is convex, then 
lip F (x) = lim sup^ calm F (x). 

X 

Proof. To simplify notation, let k := lim sup 3. ^.^ calmF (x). 

X 

(a) For any e > 0, we can find a point x^ such that |x — Xe| < e and 
calmF (xe) > k — e. Then we can find a point x^ such that |xe — x^l < e 
and \F (x^) — F (xe)| > (k — e) \xe — Xf\. As e can be made arbitrarily 
small, we have k < lip F (x) as needed. 
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(b) For every e > , there is some neighborhood of x, say B5 [x), such 
that 

calmF (x) < K + e if x e B5 (x) n X 

For any 2; G {x) fl X, consider the line segment joining y and z, which 
we denote \y, z\. As calmF (5) < k + e for all x E [y, z], there is a neigh- 
borhood around x, say V^, such that \F (x) — F {x)\ < {k + 2e) |a; — x\ for 

all X e n X. 

As [y, z] is compact, choose finitely many x such that the union of T4 
covers [y, z] . We can add y and 2; into our choice of points and rename 
them as Xi, . . . , in their order on the line segment [y, z], with xi = y 
and Xk — z. Also, we can find a point xi between xi and Xj+i such that 
Xi e V'xi n Vx^^^. Therefore, we add these Xi into xi,...,Xk and get a new 
set , . . . , again in their order on the line segment and xi — y,XK — z. 

We have: 

K-l 

\Fiy)-Fiz)\ < J2\F{x,)-Fix,+,)\ 

1=1 

K-l 

< ^(«; + 2e)| 

■^i -^i+l I 

i=l 

< {K + 2€)\y-z\, 

and as e is arbitrary, lip F (x) < k, as claimed. □ 

Convexity is a strong assumption here, but some analogous condition is 
needed, as the following examples show. 

Example 2.3. (a) Consider the set X c R defined by 



i=l >- 



X=IU 

and define the function F : X 

F(x)^ 



1 2 
by 



U{0}, 



± if±<x<- 
ifa; = 0. 



It is clear that calmF (x) = for all x G X\ {0} since F is constant on 
each component of X, and calm F (0) = 1. But 



lipF(O) = lim 



lim 



J_ 2_ 

1 



3' 



2 

3i+i 



= 2. 
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Thus, lim sup^^^o calm F (x) < lip F (0) . 

(b) Consider X C defined by X := {{xi,X2) \ x^ = xf} and the 
function F : — > R defined by F {xi,X2) = X2- One can easily check 
that lim sup^,^o calm F (x) = and lip F (0, 0) = 1 . This is an example of 
a semi-algebraic function where inequality holds. o 

Note that calmF (x) can be strictly smaller than lipF (x) even if X is 
convex, as demonstrated below. 

Example 2.4. (a) Consider F : R ^ M defined by 

Ffx) = I ^ if a; = 0, 

^ ' x^ sin otherwise. 

Here, calm F (0) = 0, but lip F (0) = oo. 
(b) Consider F : R^ ^ R defined by: 

if xi < 

irr^ ^■\ - I ifO<xi <X2/2 

l^\Xx,X2)-\ ^ ifO<Xi<-X2/2 

if xi > |x2| /2. 

We can calculate calm F (0, 0) = 2/\/b, and lipF (0, 0) = 2, so this gives 
calmF (0, 0) < lipF (0, 0). This is an example of a semi- algebraic func- 
tion where inequality holds. o 

At this point, we make a remark about subdifferentially regular functions. 
We recall the definition of subdifferential regularity. 

Definition 2.5. [|23l Definition 8.3] Consider a function / : R" RU {oo] 
and a point x with / (x) finite. For a vector v G R", one says that 

(a) t> is a regular subgradient of / at x, written v & df (x), if 

f (x) > / (x) + (f , X — x) + o (|x — x|) ; 

(b) t> is a (general) subgradient of / at x, written v E Of (x), if there are 
sequences x'^ ^ x and v'^ G df {x'^) with v'^ v and / (x'') f (x). 

(c) If / is Lipschitz continuous at x, then / is subdifferentially regular if 
df (x) = df (x). 

Though the definition of subdifferential regularity differs from that given 
in [|23l Definition 7.25], it can be deduced from [23, Corollary 8.11, Theo- 
rem 9.13 and Theorem 8.6] when / is Lipschitz, and is simple enough for 
our purposes. Subdifferentially regular functions are important and well- 
studied in variational analysis. The class of subdifferentially regular func- 
tions is closed under sums and pointwise maxima, and includes smooth 
functions and convex functions. It turns out that the calmness and Lipschitz 
moduli are equal for subdifferentially regular functions. 
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Proposition 2.6. If f : M" M U {00} /5 Lipschitz continuous at x and 
subdijferentially regular there, then calm / (x) = lip / (x). 

Proof. By [l23l Theorem 9.13], lip / (x) = max{|f | | 1; G (9/ (x)}. If f G 
(9/ (x), then v E df (x), and we observe that calm / (x) > |f | because 

f{x + tv) > fix) + {v,tv)+o{\t\) 
= /(x) + |f| +o(|t|). 

Therefore calm/(x) < lip/(x) = max{|f| | v E df (x)} < calm/(x), 
which implies that all three terms are equal. □ 

3. Calmness and robust regularization 

Recall the definition of robust regularization in Definition 11.11 To study 
robust regularization, it is useful to study the dependence of (x) on e 
instead of on x. For a point x E X, define : M+ — M by 

9x (e) = fe (x) . 

To simplify notation, we write g = g^ if it is clear from context. Here are a 
few basic properties of g^. 

Proposition 3.1. For f : X —>■ M. and g^ as defined above, we have the 
following: 

(a) gx is monotonically nondecreasing. 

(b) If f is continuous in a neighborhood ofx, then g^ is continuous in a 
neighborhood ofO. 

Proof. Part (a) is obvious. For part (b), we prove the left and right lim- 
its separately. Suppose that | e. There is a sequence of x, such that 
/ (xi) = ft, {x), and |xj — x| < Cj. We assume, by choosing a subsequence 
if needed, that Umj^oo Xj = x. We have |x — x| < e, and since / is contin- 
uous, / (xj) f {x). This means that 

fe {x) > / (x) = lim ^ (x) , 

J— >oo 

which implies g (e) > lim sup-^^ g (e) . The monotonicity of g tells us that 
^(e) = lim,-|,^(e). 

Next, suppose that increases monotonically to e. Let x be such that 
/ (x) = (x), with |x — x| < e. Since / is continuous, for every 5i > 0, 
there is a ^2 > such that \f (x') — / (x)| < 61 if |x' — x| < 62. This 
means that if e — Cj < ^2, then 

(x)>/(x)-5i = /,(x)-5i. 

As 5i can be made arbitrarily small, we conclude that hmj^^ (x) = (x), 
orlim,-|,^(e) =^(e). □ 
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It turns out that calmness of the robust regularization is related to the 
derivative of g^. 

Proposition 3.2. 7/"/ : X — > M and e > 0, then calm/,, (x) < calm^f^. (e). 
If in addition X = M" and is dijferentiable at e, then 

calm/, {x) = calm^^ (e) = g'^ (e) . 

Proof. For the first part, we proceed to show that if k > calmgf^. (e), then 
K > calm (a;). If |a; — a; I < e, we have 

]B,_|5;_a;| (x) C B, (x) C B,+|i_^| (x) , 

which implies 

fe-\x-x\ (X) < fe (X) < fe+\x-x\ (x) . 

Then note that if x is close enough to x, we have 

fe (x) < fe+\x-x\ (x) = g^ {e + \x - x\) < gx (e) + k\x-x\, 
and similarly 

fe (x) > fe-\x-x\ (x) = (e - |x - x\) > gx (c) - k\x-x\, 

which tells us that | (5) — (x) | < k\x — x\, which is what we need. 

For the second part, it is clear from the definition of the derivative that 
g'^ (e) = calm (e). We prove that if k, < g'^ (e), then k < calm (x). By 
the differentiability of g^, there is some S > such that for any < S < 5, 
we have 

fe+six) = gxie + 5) 
> gx (e) + 
= f, (x) + k5. 

For any < 6 <6, there is some xs E M^^s (x) such that / (x^) = f^+s (x). 
Let Xs = {xs - x) + X. We have /, (x^) = f^+s (x), which gives 

fe (xs) — fe (x) > K,5. Slncc Xs was chosen such that 6 = |x5 — x|, we have 
fe (xs) — fe (x) > k\xs — x\, wMch implies k < calm fe (x) as needed. □ 

Remark 3.3. A similar statement can be made for e = 0, except that we 
change calmness to "calm from above" as defined in [|23l, Section 8F] in 
both parts. 

We have the following corollary. The subdifferential "9" was defined in 
Definition [231 

Corollary 3.4. 7/"/ : M" — > M, e > and g^ is Lipschitz at e, then 
calm/, (x) < lip^^ (e) = sup {|?/| \ yedg^ (e)} . 
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Proof. It is clear that calm/e {x) < calm^fa; (e) < lipgx (e). The formula 
lipgx (e)= sup{|?/| | y G dg^ (e)} follows from |l23l Theorem 9.13, Defini- 
tion 9.1]. □ 

In general, the robust regularization is calm. 

Proposition 3.5. For a continuous function f : X —>■ M., there is an e > 
such that fe is calm at x for all < e < e except on a subset of (0, e] of 
measure zero. 

Proof. By Proposition 13. If b). since / is continuous at x, gx is continuous 
in [0, e] for some e > 0. Since gx is monotonically nondecreasing, it is 
differentiable in all [0, e] except for a set of measure zero. The derivative 
g'^ (e) equals calm (x) by Proposition 13.21 □ 

Remark 3.6. In general, the above result cannot be improved. For an ex- 
ample, let c : [0, 1] [0, 1] denote the Cantor function, commonly used 
in real analysis texts as an example of a function that is not absolutely con- 
tinuous and not satisfying the Fundamental Theorem of Calculus. Then 
calmce (0) = oo for all e lying in the Cantor set.o 

4. Robust regularization in general 

In this section, in Corollary 14.61 we prove that if lip / (x) < oo for x 
close to but not equal to x, then lip (x) < oo for all small e > 0, even 
when lip / (x) = oo. To present the details of the proof, we need a short 
foray into set- valued analysis. 

Definition 4.1. [|23l Example 4.13] For two sets C,D c. W^, the Pompieu- 
Hausdorff distance between C and D, denoted by d (C, D), is defined by 

d{C, D) := M {r] > \ C C D + 7]M, D C C + 7]M} . 

Definition 4.2. [|23l Definitions 9.26, 9.28] A mapping S -.X ^W^is Lip- 
schitz continuous on its domain X C M", if it is nonempty-closed-valued 
on X and there exists k > 0, a Lipschitz constant, such that 

d {S (x') , S (x)) < k\x' — x\ for all x, x' G X, 

or equivalently, S (x') C S (x) + k \x' — x| B for all x, x' G X. The Lips- 
chitz modulus is defined as 

lip5(x) :=limsup ^^^,^f^'^^"^\ 

X 
x^x' 

and is the infimum of all k such that there exists a neighborhood U of x 
such that 5* is Lipschitz continuous with constant n'mU D X.o 
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For F : X — > R*", we may write the robust regularization : X ^ 
by = F o where : X =^ X is defined by $e {x) = B, {x) n X. 
For reasons that will be clear later in Section |71 we consider the extension 
: R*" ^ X defined by (x) = B, (x) n X. It is clear that \x= $e 
using our previous notation, and it follows straight from the definitions that 
lip $e (x) < lip <le (x) for X G X. 

Definition 4.3. We say that X C R" is peaceful at x e X if lip $e (x) is 
finite for all small e > 0. If in addition lim sup^^Q lip $e (x) < k for all 
small e > 0, we say that X is peaceful with modulus k at x, or n-peaceful 
at X. 

When X lies in the interior of X and e is small enough, then $e is Lips- 
chitz with constant 1. In section |71 we will find weaker conditions on X for 
the Lipschitz continuity of We will see that convex sets are 1-peaceful, 
but for now, we remark that if X is convex, then $e is globally Lipschitz in 
X. 

Proposition 4.4. IfX is a convex set, then $e (x) C $ (x') + |x — x'| B/or 
all X, x' G X. 

Proof. The condition we are required to prove is equivalent to 

B, (x) n X C (B, (x ) n X) + |x - x'l B for x, x' G X. 

For any point x G B^ (x) fl X, the line segment [x', x] lies in X, and is 
of length at most |x — x| + |x — x'|. The ball B^ (x') can contain the line 
segment [x', x], in which case x G B^ (x') fl X, or the boundary of B^ (x') 
may intersect [x', x] at a point, say x. Since X is a convex set, we have 
X G Be (x') n X. Furthermore 

X' rir> I I />' nn^ I f: 

< |x — x| + |x — x'l — e 

< |x — x'l , 

SOX G (B, (x') nX) + |x-x'|B. □ 

We remark that if X is nearly radial at x as introduced in [fT5l , then X is 
1-peaceful: see Section |71 The set X is nearly radial at x if 

dist (x, X + Tx (x)) — » as X X in X. 

The set X is nearly radial if it is nearly radial at all points in X. The no- 
tation Tx (x) refers to the (Bouligand) tangent cone (or "contingent cone") 
to X at X G X, formally defined as 

Txi^) = {limt~^(xr — x) : tr I 0, x^ — x, x^ G X} 
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(see, for example, 11231 Definition 6.1]). Many sets are nearly radial, includ- 
ing for instance semi-algebraic sets, amenable sets and smooth manifolds. 

We now present a result on the regularizing property of robust regular- 
ization. In Proposition 14.51 below, condition (i) allows us to evaluate the 
Lipschitz modulus of functions whose domains are not necessarily convex. 
One situation where (i) is interesting is when X is a smooth manifold. 

Proposition 4.5. For F : X M"*, suppose that either ( i) or ( ii) holds. 

( i) X is peaceful and lip F (x) < oo for all x close to but not equal to x. 
Here, F : R" ^ R™ is an extension of F on R" such that F\x = F. 

(ii) X is convex and lipF (x) < oc for all x E X close to but not equal 
to X. 

Then lip F^ (x) is finite for all small e > 0. 

Proof. The proof for both conditions are similar, so they will be treated 
together. One notes that lip F (x) < lip F (x) always by the definition of 
these Lipschitz moduli, so we assume lip F (x) < oo for all x G X close to 
but not equal to x until we have to distinguish these cases. 

First, we prove that lipF : X — R+ is upper semicontinuous. This 
result is just a slight modification of the first part of [23, Theorem 9.2], 
but we include the proof for completeness. Suppose that Xj — > x. By the 
definition of lip F, we can find Xj,i, Xj,2 G X such that 

- — , > lipF (xi) - Xj - X , 

and |xjj — Xj| < 
Taking limits as i — > oo, we see that Xi 

lip F (x) > lim sup 

= lim sup lip F (xi) . 

j— >oo 

Thus lipF : X ^ R+ is upper semicontinuous. 

So for ei small enough, choose 62 < ei such that lip F is bounded above 
in Ci = (B^i+ej (x) \IBe^_e2 (x)) n X, say by the constant ki. Then for any 
K2 > Ki and any x E Ci, there is an e^. such that F is Lipschitz continuous 
on B^^ (x) n X with constant K2 with respect to X. Thus U^^Ci {^e^ (x)} 
is an open cover of Ci. 

By the Lebesgue Number Lemma, there is a constant S such that if xi, X2 
lie in Ci and |xi — X2I < S, then the line segment [xi, X2] lies in one of the 
open balls B^^ (x) for some x E Ci. We may assume that 5 < £2. 

Also, since X is peaceful at x, choose ci small enough so that lip (x) 
is finite, say lip $^1 (x) < K. If X is convex, then this is possible due to 





forj = l,2. 




■ X, and it follows that 




-F{x,,2)\ 




- Xi^2\ 
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Proposition 14.41 We can assume that K > 2. Therefore, there is an open 
set U about x such that is Lipschitz in ?7 fl X with constant K, that is 

(x) C {x') + K\x-x'\M for all x,x' eUnX. 
So, for X, G f/ n B i ix) n X, we want to show that 



2K 



Suppose that y G F^^ (x). So y = F (x) for some x G B^^^ (x) fl X. 
If X G Bei-5|, ^^'^ ^ ^ ^ ^ because |x' - x| < So 

2/ G (x'). Otherwise x G (Kei+^ (^) V^^i-^i? '^^O ^ 

We have (x) C (x') + |x - x'| B. So there is some x G (x') 
such that 

1^ , , /, (5 (5 

X — X < K \x — X \ < K = -. 

I I - I ^ - 2K 2 



Furthermore, 

^ , , , ,^ , ^ ^ 35 

x-x| < |x-x| + |x-x| + |x-x| <ei + — + - < ei + — < ei + €2, 

2K I 4 



and 

X — X > X — X — X — X — X — X > Ci > ei > ei — Co. 

' - 2K 2- ^ 

Hence x G (B^j+^j (^) (2;)) H X. We now proceed to prove the 

inequality |F (x) — F (x)| < /t2 |x — x| for the two cases. 

Condition (i): Since |x — x| < (5, the line segment [x, x] lies in B^^ (x) for 
some X G X. Since the line segment [x, x] is convex and lipF is bounded 
from above by K2 there, we have 



IF fx) - Fix] 



F{x)-F (x) 



< fi;2 X — X 



by 1231, Theorem 9.2]. 

Condition (ii): The proof is similar, except that [x, x] C X, and lip F is 
bounded above by K2. 

On establishing |F (x) — F (x)| < /t2 |x — x|, we note that 

F(x) G F (x) + K2 |x - x| B 
C F,j (x') + fi:2 |x - x|B 
C F,i(x) + i^/t2|x-x|B, 

and we are done. □ 

We are now ready to relate lip /e (x) to lip / (x). We remind the reader 
that in the proof of Corollary 14 . 61 below, : X =^ M is a set- valued map as 
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introduced in Definition ll.il which is similar to but maps to intervals in 
M. 

Corollary 4.6. For f : X ^ M., if either condition ( i) or condition ( ii) in 
Proposition \4. 5[ for F : X — > M taken to be f holds, then lip (x) < oo for 
all small e > 0. 

Proof. By Proposition 14.51 we have lip (x) < oo with the given condi- 
tions. It remains to prove that lip/e (x) < lip/e (x). We can do this by 
proving that lip S (x) < lip S (x), where S* : X =^ M is a set- valued map, 
and : X ^ M is defined by S (x) = sup {y \ y e S (x)}. Note that if 
S = f^thonS ={!:) = U 

For any k > lipS* (x), we have d (S (x) , S (x)) < k |x — x| for x, x G X 
close enough to x by fIT, Definition 9.26]. The definition of the Pompeiu- 
Hausdorff distance tells us that S (x) C S (x) + k\x — x\, which implies 
S (x) < S* (x) + K |x — x|. By reversing the roles of x and x, we obtain 
\S (x) — S {x)\ < K |x — x|. So K > lip 5 (x), and since k is arbitrary, we 
have lip S (x) < lip S (x) as needed. □ 

5. Semi-algebraic robust regularization 

In this section, in Theorem l5.3l we prove that if / : M" — M is continu- 
ous and semi- algebraic, then at any given point, the robust regularization is 
locally Lipschitz there for all sufficiently small e > 0. This theorem is more 
appealing than Corollary I4.6l because the required condition is weaker. The 
condition lip / (x) < oo for all x close to but not equal to x in Corollary 14.61 
is a strong condition because if a function is not Lipschitz at a point x, it is 
likely that it is not Lipschitz at some points close to x as well. For example 
in / : ^ M defined by / (xi,X2) = |v^|, / is not Lipschitz at all 
points where xi = 0. 

We proceed to prove the main theorem of this section in the steps outlined 
below. 

Proposition 5.1. For / : X — > M, where X C is convex, define G : 

X X M+ ^ M+ U {oo} by 

G (x, e) := lim sup lip (x) . 

€ — 

Iff is semi-algebraic, then the maps (x, e) i-^ calm (x), (x, e) t-^ lip (x) 
and G are semi-algebraic. 

Proof. The semi-algebraic nature is a consequence of the Tarski-Seidenberg 
quantifier elimination. □ 

The semi-algebraicity of (x, e) i— calm (x) gives us an indication of 
how the map e i-^ calm (x) behaves asymptotically. 
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Proposition 5.2. Suppose that f : X ^Wis continuous and semi-algebraic, 
where X C M". Fix x e X. Then calm/g (x) = o (^^^ as e \ 0. Hence 
is calm at xfor all small e > 0. 

Proof. The map is semi- algebraic because it can be written as a com- 
position of semi-algebraic maps e i-^ (x, e) ^-^ (x) . Thus is dilfer- 
entiable on some open interval of the form (0, e) for e > 0. Recall that 
calm (7^. (e) = g'^ (e) by Proposition 13 .21 

We show that for any K > 0, we can reduce e if necessary so that the map 
e 1—*^ calm/e (x) is bounded from above by e t-^ on e G [0, e]. For any 
K > 0, there exists an e > such that either g'^ (e) < for all < e < e, 
or g'^ (e) > for all < e < e. The latter cannot happen, otherwise for 
any < e < e, 



This contradicts the continuity of gx- If e is small enough, the derivatives 
of gx exist for all small e > and g'^ (e) = calm/^ (x) by Proposition 13.21 



Consider / : [0, 1] ^ M defined by / (x) = x^/''. Then go (e) = e^/\ so 
calm/f (0) = g'o (e) = le^^/''^^^ As /c ^ oo, we see that the bound above 
is tight. 

We are now ready to state the main theorem of this paper. In the particular 
case of X = M", we have the following theorem. 

Theorem 5.3. Consider any continuous semi-algebraic function f : M" — > 
M. At any fixed point x G M", the robust regularization is Lipschitz at x, 
and its calmness and Lipschitz moduli, calm (x) and lip (x), agree for 
sufficiently small e and behave like o (i) as e J. 0. 

Proof. In view of Proposition 15. 2[ we only need to prove the there is some 
e > such that lip/e (x) = calm/^ (x) for all e G (0,e]. We can as- 
sume that gx is twice continuously differentiable in (0,e]. The graph of 
G : M" X ]R+ — > R_,_ as defined in Proposition 15. II is semi-algebraic, so by 
the decomposition theorem [10. Theorem 6.7], there is a finite partition of 
definable manifolds Ci, . . . , C; such that G is C^. 




This gives us the required result. 



□ 



LIPSCHITZ BEHAVIOR OF THE ROBUST REGULARIZATION 15 

If the segment {x} x (0, e] lies in the (relative) interior of one definable 
manifold, then 

lip/j(x) = lim sup calm (x) (by Proposition |Z2|) 
= hmsup(y'^(e) (by Proposition |3^ 

a;— »5; 

= calm /, (x) , 

and we have nothing to do. Therefore, assume that the segment is on the 
boundary of two or more of the Cj. 

Since G is semi-algebraic, the map e lim sup^,,^^ lip fa (x) is semi- 
algebraic, so we can reduce e > as necessary such that either 

(1) limsupo.^^^ lip fa (x) < calm/e (x) for all e G (0, e], or 

(2) limsupo,.^^ lip fa (x) = calm/^ (x) for all e G (0, e], or 

(3) limsupo,.^^ lip fa (x) > calm/^ (x) for all e G (0, e]. 

Case (1) cannot hold because lip (x) > calm/^ (x). Case (2) is what 
we seek to prove, so we proceed to show that case (3) cannot happen by 
contradiction. 

We can choose e, Mi, M2 > such that < e < e and 

calm (x) < M2 < Ml < lim sup lip fa (x) for all e G [e, e] . 

We state and prove a lemma important to the rest of the proof before con- 
tinuing. 

Lemma 5.4. There exists an interval (ei, €2) contained in (e, e] and a man- 
ifold Ti C M" X such that 

(1) {x} X (ei,e2) Ccl (Ti). 

(2) Ti is an open manifold. 

(ij i7 : R" X M+ ^ M, defined by H (x, e) = /, (x), is in Ti. 

(4) For all (x, e) G Ti, we have Mi < g'^ (e) < 00. 

(5) (x, e) I— i> g'^ (e) is continuous in Ti. 

Proof. Consider the set 

T:={(x,e)|Mi<(7;(e)<oo}. 

First, we prove that {x} x [e, e] C cl T. It suffices to show that for all 
e G (e, e], (x, e) G cl T. This can in turn be proven by showing that for 
all 5 > 0, we can find x', e' such that |x — x'| < S, |e — e'| < 5 such that 
(x', e') G T, or equivalently. Mi < g'^, (e') < 00. 

Since lim sup^,^^ lip fa (x) > Mi, there is some e° such that |e° — e| < | 
and lip f^o (x) > Mi. 
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limsup \dgx (e°)| > lim sup calm /^o (x) = lip fe° (x) , 

there is some x' such that |a; — a;'| < 5 and \dgx' (e°) | > |lip fe° (x) + |Mi. 

Finally, since gx' {■) is semi- algebraic, we can find some e' such that 
\e' — e°\ < |, g'^, (e') is well defined and finite, and 



g',, (eO > \dgx' (e°)| - ^ (lip/,o (x) - Mi) > M,. 



This choice of x' and e' are easily verified to satisfy the requirements stated. 

By the decomposition theorem (10', Theorem 6.7], T can be decomposed 
into a finite disjoint union of smooth manifolds Ti, T2, Tp on which 
H is C^. Since {x} x [e, e] C cl T, there must be some Tj and (ei, 62) such 
that {x} X (d, 62) C cl Tj. Without loss of generality, let one such Tj be Ti. 

Conditions (1), (2), (3) and (4) are automatically satisfied. Note that 
g'^ (e) is exactly the derivative of H {■,■) with respect to the second coor- 
dinate, and so Property (5) is satisfied. This concludes the proof of the 
lemma. □ 



We now continue with the rest of the proof of the theorem. If Ti is of 
dimension one, then we have Ti D {x} x (ei, 62). Recall that if the deriva- 
tive g'g (e) exists, then g'^ (e) = calm (x) by Proposition [X2l This would 
mean that calm/^ (x) > M2, which contradicts our earlier assumption of 
calm (x) < M2. Therefore, the manifold Ti is of dimension at least two. 

Using Lemma 15.71 which we will prove later, we can construct the map 
(f : [0, 1) X (ei, €2) clTi, such that its derivative with respect to the second 
variable exists and is continuous, and ip (0, e) = (x, e) for all e E (ei, 62). 

For each < S < 1, consider the path xs '■ [h,^2] defined by 

Xs (e) := (5, e). We have 

fe2 {xs (£2)) - /ei {xs (ei)) 

VH {xs (s) , s) ■ (x's (s) , 1) ds 



£2 



VxH {xs (s) , s) ■ x's (s) ds+ VsH {xs (s) , s) ds 
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where H (x, e) = (x). The second component of VH (xs (s) , s) is sim- 
ply g'xg(s) i-^)- fi^^*- component can be analyzed as follows: 

V^H {xs (s) , s) ■ x's (s) 

= lim ^ {H {xs (s) + tx'g (s) ,s) - H {xs (s) , s)) 

= S i ^-^^ ^ '^^'^ ~ ^^"^"^"^ ■ 

Provided that t < s, (x^ (s)) C B, {xs (s) + tx'^ (s)), and 

so 

Vx.iJ {xs (s) , s) ■ x's (s) 

- !^ ^ {^s-t\x',(s)\ (^)) - (^))) 

Hence, 

/e2 {X5 (e2)) - /ei (x<5 (ei)) 

/ V^iJ {xs (s) , s) • x'^ (s) c/s + / V^-ff (X5 (s) , s) c/s 

(1- I4(s)l)^£,(s) (^J) 



f2 



> 

Since the derivatives of are continuous, x'^ (s) ^ Xq (s) = as 5 ^ for 
ei < s < 62- In fact, the term \x'^ (s) | converges to zero uniformly in [ei, 62]. 
To see this, recall that x'^ (s) is a partial derivative of 93. Since is C^, x'^ (s) 
is continuous with respect to s and 5. For any /? > and s G [ei, 62], there 
exists 7s such that 

\x'^ (5)1 < /3 if 5 < 7s and |s — s| < 7^. 

The existence of 7 such that 

\x'^ {s)\ < /3 if 5 < 7 and s G [ei, £2] 

follows by the compactness of [ei, €2]. So we may choose 5 small enough 
so that 

(1 - K > for all s G [ei, £2] . 

Now, for 5 small enough and i = 1,2, we have g'^ (ii) < M2, so this 
gives us calm/e- (x) = g'^ (q) < M2 by Proposition 13 .21 Therefore, if 5 is 
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small enough, 

{XS (Ci)) - /e^ (x)| < M2 \XS (Ci) - x\ . 

Recall that if the derivative g'^ (e) exists, then g'^ (e) = calm (x) by 
Proposition 13 .21 On the one hand, we have 

(x) - (x) = r g'As)ds< r M2ds = M2{i2-h). 

J e\ J ei 

But on the other hand, xs (s) G Ti for < 5 < 1, and so fi'^^(s) (s) > Mi 
by Lemma [54l If 5 is small enough, we have 

\fe2 (^) - fh 

> \fe2{S:5{e2)) - fiA^s{ei))\ 

- (|/e2 (5<5 (^2)) - fi2 (^)| + (^5 (ei)) - /ei (^)|) 

> r {1 - \x's is)\) g',^^,) (s) ds 

-M2{\xs {e2)-x\ + \X5 {ei)-x\) 

> / {l-\x's{s)\)Mids-M2{\x5{e2)-x\ + \xs{ei)-x\) 

n /Ml + M2\ 

> / 7: - M2 (€2) - x| + |x5 (ei) - x|) 



2 

1 (e2 - ei) - M2 {\xs (ea) - x\ + \xs (ei) - x\) . 

As 5 is arbitrarily small and the terms |x5 (q) — x| as 5 for 
i = l,2, we have l/^^ (x) - (x) | > (Mi±A^j (^-^ - ei). This is a contra- 
diction, and thus we are done. □ 

Before we prove Lemma 15.71 below, we need to recall the definition of 
simplicial complexes from [|1U Section 3.2.1]. A simplex with vertices 
ao, . . . , Orf is 

[ao, . . . , = {x G M" I 3Ao, . . . , G [0, 1] , 

d d 

Aj = 1 and x = ^ \ai.} 

i=0 i=0 

The corresponding open simplex is 

(ao,...,ad) = {x G M" I 3Ao,...,Ad G (0,1), 



Aj = 1 and x = ^ Ajaj.} 

i=0 i=0 
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We shall denote by int (cr) the open simplex corresponding to the simplex 
o. A face of the simplex o = [oq, . . . , a^] is a simplex r = [60, • • • , fee] such 
that 

{60, . . . ,fee} C {ao, . . . ,ad} . 

K finite simplicial complex in M" is a finite collection K = {ai, . . . , cTp} 
of simplices cTj C M" such that, for every ctj, aj E K, the intersection aiCiaj 
is either empty or is a common face of ctj and a-,. We set ji^'l = Uai^KO'i; 
this is a semi-algebraic subset of M". We recall a result on relating semi- 
algebraic sets to simplicial complexes. 

Theorem 5.5. [fTTl Theorem 3.12] Let S C be a compact semi-algebraic 
set, and Si, . . . , Sp, semi-algebraic subsets of S. Then there exists a fi- 
nite simplicial complex K in M" and a semi-algebraic homeomorphism 
h : \K\ ^ S, such that each Sk is the image by h of a union of open 
simplices of K. 

We need yet another result for the proof of Lemma [5771 

Proposition 5.6. Suppose that : (0, 1)^ R, not necessarily semi- 
algebraic, is continuous in (0, 1)^. Let gph0 C (0, 1)^ x M Z^e the graph of 
(j). Then for any t G (0, 1), cl (gph 0) fl (0, t) x R is either a single point or 
a connected line segment. 

Proof. Suppose that ((0, t) , ai) and ((0, t) , 02) lie in cl (gph0). We need 
to show that for any « G (ai, 02), ((0, t) , a) lies in cl (gph 0). 

For any e > 0, we can find points pi,p2 G (0, 1)^ such that the points 
(pi,ai) , (^2,02) £ gph0 are such that |aj — ai\ < e and \pi — (0,t)| < e 
for i = 1,2. Recall that by definition = 4>{pi) for i = 1,2. Choose 
e such that di + e < 0.2 — e. By the intermediate value theorem, for any 
a G (ai + e, 0.2 — e), there exists a point p in the line segment [pi,P2] such 
that0(p) = a. Moreover, \p—{0,t)\ < maxj=i^2 b,; — (0, Letting 
e — *• 0, we see that ((0, t) ,a) G cl (gph 0) as needed. □ 

We now prove our last result important for the proof of Theorem 15.31 
The proof of the lemma below is similar to the proof of the Curve Selection 
Lemma in [fTTl Theorem 3.13]. 

Lemma 5.7. Let S G be a semi-algebraic set, and r : [ei , £2] ^ be a 
semi-algebraic curve such that r ([ci, £2]) H 5 = and r ([ci, £2]) C cl (S). 
Then there exists a function ip : [0, 1] x [ci, 62] ^"'> with [ei, £2] 7^ and 
[ii, 62] C [ei, 62], such that 

(1) ip{0,e) = T{e)fore G [61,62] and ^{{0,1] x [61,62]) C S. 

(2) The partial derivative ofip with respect to the second variable, which 
we denote by -^(f, exists and is continuous in [0, 1] x [ei, 62]. 
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Proof. Replacing S with its intersection with a closed bounded set contain- 
ing r([ei,e2]), we can assume S is bounded. Then c\{S) is a compact 
semi-algebraic set. By Theorem 15. 5 1 there is a finite simplicial complex K 
and a semi-algebraic homeomorphism h : \K\ cl {S), such that S and 
r ([ei, £2]) are images by /i of a union of open simplices m K . In partic- 
ular, this means that there is an open interval (61,62) C [61,62] such that 
r ((ei, 62)) is an image by /i of a 1-dimensional open simplex in K. Since 
h^^ o r ((ei, £2)) is in cl {S) but not in S, there is a simplex a oi K which 
has h^^ o r ([ci, 62]) lying in the boundary of a, and h (int (a)) C S. 
Let 0" be the barycenter of a. Define the map 5 : [0, 1] x [ei, 62] — > by 

5(t,e) = (1 -t) /i"^ or(e) + ta. 

The map above satisfies 5 ((0, 1] x (ei, 62)) C int (a). By contracting the 
interval [ei, 62] slightly, ip = ho 5 satisfies property (1). 

By contracting the interval [ei, 62] if necessary and applying the decom- 
position theorem [fTOl Theorem 6.7], we can assume that Lp is in the set 
(0, t\ X [ei, €2] for some i G (0, 1). 

Since r is semi-algebraic, we contract the interval [ei, 62] again if neces- 
sary so that r is there. Therefore, ^(/? exists in [0, t] x [ci, 62]. It remains 
to show that is continuous in [0, t] x [ei, 62]. We do this by showing that 
: [0, t] X [ei, 62] M, the zth component of the derivative with respect 
to the second variable, is continuous for each i. 

Since ^^ipi is continuous in (0, t\ x [ci, 62], it remains to show that it is 
continuous at every point in {0} x [ei, 62] . The graph of ^^ipi corresponding 
to the domain (0, t\ x [ei, €2], which we denote by gph (^<^j) , is a subset of 
(0, t] X [ei, ea] x M. We show that ((0, e) , (0, e)) e cl (gph (|</^0) . 
For small ti, t2 > 0, consider Lpi (ti, e — and (p>i {ti,e + ^2)- By the in- 
termediate value theorem, there is some e G (e — t2, e + ^2) such that 

— (ti, e) = — {^i [ti, e + t2}- (pi (ti, e - t2)) • 
ae 2t2 

If ^2 were chosen such that 

1 d 
— {ipi (0, e + t2) - ipi (0, e - t2)) - (0, ( 
2t2 oe 

is small and ti is chosen such that 

(V9i (ti, e + ^2) - 'P^{tl,e-t2)) - ^ {(pt (0, e + t2) - V^i (0, e - ^2)) 

ZI2 ^t2 

is small, then \ -§^^i (ti, e) — ^v^i (0, e)| is small. Taking ^2 — > and ti 
0, we have ((0, e) , ^-^ipi (0, e)) G cl (gph (^v^i)) as desired. 
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Recall that the graph gph {■§^'^1) is taken corresponding to the domain 
(0, t\ X [ei, 62], and is a manifold of dimension 2 in M^. Its boundary is of 
dimension 1 |lli Proposition 3.16], so the intersection of cl (gph {-§^^i)) 
with {0} X [61,62] X M is of dimension 1 as well, and is homeomorphic 
to a closed line segment. There cannot be an interval [ci, £2] C [ci, £2] on 
which cl (gph (£<^i)) ^ {0} x {e} x ^ has more than one value for all 
e G [^15^2] because by appealing to Proposition 15. 6[ this implies that the 
dimension cannot be 1. We note however that it is possible that there exists 
an e G [ei, £2] such that cl (gph {^^^i)) n {0} x {e} x M is a 1-dimensional 
line segment. This can only happen for only finitely many e G [ei, £2] due 
to semi-algebraicity. 

In any case, we can contract the interval [ei,e2] if necessary so that 
cl (gph {§^(pi)) n {0} X {e} X R is a single point for all e G [ei, 62]. This 
means that for any {t, e) (0, e), we have ^^Pi (t, e) 



lishing the continuity of ^ipi 
allows us to assume that t = 1 



-,■) on [0,t] X [61,62] 
and we are done. 



^V^i (0, e), estab- 
A reparametrization 

□ 



6. Quadratic examples 

In this section, we show how the robust regularization can be calculated 
for quadratic examples, which are more-or-less standard in the spirit of 
jSl [11 . We write A ^ for a real symmetric matrix A if yl is positive 
semidefinite. 

Theorem 6.1. (Euclidean norm) For any real m x n matrix A and vector 
b G M™, consider the function (7 : M" ^ M defined by 

g{x) = \\Ax + 6II2, 

Then the following properties are equivalent for any point (x, t) G M" x M.- 

(i) t > (x) 

(ii) there exists a real ^ such that 

tim Ax + b eA 
{Ax + b)^ t~ fi 
eA^ 



>- 0. 



Proof. Applying [1 , Thm 4.5.60] shows t > g^ix) holds if and only if there 
exist real s and /i satisfying 



{Ax + bf 



t - s 
Ax + b eA 



s — fi 








> 

y 0. 



and the result now follows immediately. 



□ 
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Since the matrix in property (ii) above is an affine function of the vari- 
ables X, t and jji, it follows that the robust regularization is "semidefinite- 
representable", in the language of [[T]|. This result allows us to use in 
building tractable representations of convex optimization problems as semi- 
definite programs. 

An easy consequence of the above result is a representation for the robust 
regularization of any strictly convex quadratic function. 

Corollary 6.2. (quadratics) For any real positive definite n-by-n matrix H, 
vector c G R", and scalar d, consider the function /i : M" — ^> R defined by 

h{x) = x^Hx + 2c^x + d. 

Then the following properties are equivalent for any point {x, t) G M" x R.- 

(i) t > K{x); 

(ii) there exist reals s and fi such that 

t- + c^H~^c-d > 
sin H^/^x + H'/^c tH^/^ 

[H^'^x + H^^'^cf s-fi ^0. 



Proof. Clearly t >h^ (x) if and only if 

\\y-x\\2<e \\H^^'^y + H-^/^c\\l<t-d + c^H-^c. 
This property in turn is equivalent to the existence of a real s satisfying 

< t~d + c^H'^c and 
\\y-x\\2<e \\H^^^y + H'^^^cy < s, 

and the result now follows from the preceding theorem. □ 
Since the quadratic inequality 

t-s'^ + c^H-^c -d>0 
is semidefinite-representable, so is the robust regularization he. 

7. 1-PEACEFUL SETS 

In this section, we prove that X C R" is nearly radial implies X is 
1-peaceful using the Mordukhovich Criterion [23, Theorem 9.40], which 
relates the Lipschitz modulus of set-valued maps to normal cones of its 
graph. The next section discusses further properties of nearly radial sets 
and how they are common in analysis. 

The Mordukhovich Criterion requires the domain of the set-valued map 
to be M", so^we recall the map : M" ^ M" by (x) = B, (x) n X. 
Recall that = $e and lip$, (x) < lip$, (x) for all x e X. Let us 
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recall the definitions of normal cones, the Aubin property and the graphical 
modulus. 

Definition 7.1. El Definition 6.3] Let X c M" and ^ G X. A vector 

V is normal to X at x in the regular sense, or a regular normal, written 

V e Nx (x), if 

{v,x — x) < o{\x — x\) for a; G X. 
It is normal to X at x in the general sense, or simply a normal vector, 
written v E Nx {x), if there are sequences x" — ^ x and — ^ v with 

v"" e Nxix''). 

Definition 7.2. |l23l Definition 9.36] For X C M", a mapping 5 : X =^ 
has the Aubin property at x for u, where x E X and u E S (x), if gph S 
is locally closed at (x, u) and there are neighborhoods of x and W of u 
such that 

S (x') nW C S{x) + k\x' -x\3 for all x, x' G X n V. 
The graphical modulus of S at xfor u is 

lipS'(x|M) := inf{/t I There are neighbourhoods 

Vofx,Wofu such that 

S {x) r}W E S [x) + k\x' - x\M 
forallx,x' G XnF}. 

If 5 is single- valued at x, then in keeping with the notation of lip in Defini- 
tion l2.1l we write lip S (x) instead of lip S [x \ S (x)). Note that this equals 
lip S (x) if S is continuous at x. o 

A set-valued map S is locally compact around x if there exist a neigh- 
borhood y of X and a compact set C C F such that S {V) C C. This is 
equivalent to S {V) being a bounded set, which is the case when S is outer 
semicontinuous and S (x) is bounded. If S is outer semicontinuous and lo- 
cally compact at x, then by [|20l Theorem 1.42], the Lipschitz modulus and 
the Aubin property are related by 

lip 5* (x) = max {lip S {x\u)} . 

u^S{x) 

In finite dimensions, we need S (x) to be bounded and S to be outer semi- 
continuous for the formula above to hold. 
Here is a lemma on convex cones. 

Lemma 7.3. Given any two convex cones Ci and C2 polar to each other 
and any vector x, we have 

{d{x,Ci)f + id{x,C2)f = \\xf 
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Proof. This is a simple consequence of [|23l Exercise 12.22] 



□ 



We now present our result on the relation between 1-peaceful sets and 
nearly radial sets. 

Theorem 7.4. IfX is nearly radial at x, then X is 1-peaceful at x. The con- 
verse holds ifX is subdifferentially regular for all points in a neighborhood 
around x. 



Proof. The graph of is the intersection of M" x X and the set D C 
W X W defined by 



By applying a rule on the normal cones of products of sets [|23l Proposition 
6.41], we infer that A^r^xx {x, y) = {0} x Nx (y)- Define the real valued 
function (^o : M" x M" ^ M_,_ by gQ{x,y) := ^\\x — y\f. Then the gradient 
of 5^0 is VfifQ {x,y) = {x-y,y- x). 

From this point, we assume that ||x — ?/|| = e. The normal cone of D 
at {x, y) is No {x, y) = {{x — y, y — x)} using |l23l Exercise 6.7]. On 
applying a rule on the normal cones of intersections [|23l Theorem 6.42], we 



(7.1) iVgpi,^^ (x, y) C ({0} X Nx (y)) + M+ {(x -y,y-x)}. 



Furthermore, if X is subdifferentially regular at y, the above set inclu- 
sion is an equation. By the Mordukhovich criterion Il23l Theorem 9.40], 
$e has the Aubin Property at (x, y) if and only if the graphical modulus 
lip $e {x I y) is finite. It can be calculated by appealing to the formulas for 
the coderivative D* [23 1 Definition 8.33] and outer norm |-|^ ll23l Section 
9D] below. 



D ■■= {{x,y) I \\x - y\\ < e} . 



get 



+ 



lip <l>e (x I y) 



D*^, {x I y) (by [[231 Theorem 9.40]) 



sup sup 1 1 2; 1 1 (by Il23l Section 9D1) 



sup < ||2;|| I {w, z) G gphZ}*<l>e, \\w\\ < 1 



(7.2) 



= sup 1 1|2;|| I {-z, w) e A/'gph$, {x, y) , \\w\\ < 1 
(by [111 Definition 8.33]) 

< sup{||z|| \{-z,w)e{{0}xNx{y)) 
+M+ {{x -y,y-x)}, \\w\\ < 1. 



We can assume that z = y — x with a rescaling, and w = y — x + v for some 
V G Nx (y). Since ({0} x Nx {y}) + {{x — y,y — x)} is positively 
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homogeneous set, we could find the supremum of in the same set and 
the formula reduces to 

\\y - A\ 



lip $e {x \y) < sup 
sup 

V 

(7.3) 



v&Nx{y) \\y-X + v\ 

\\x - v\ 



\\x - y\\ 



d{x-y,Nx{y)) 

d(x-y,Nx{y)) 

we prove that for any open set W about x, we have 



For a fixed x ^ y, say x, we have 1/lip $e [x \ y) > m-L n • First, 

ii"^ y\\ 



(7 4) inf d{x-y,Nx{y)) _ .^^ d{x-y,Nx{y)) 

y(^wnx \\x — y\\ yew(\x \\x — ■yll 

y^x II ^11 y^x II ^11 

It is clear that "<" holds because Nx iy) C Nx (y), so we proceed to prove 
the other inequality. Consider d{x — y, Nx (y))- Let v E PnxIv) {x — y), 
the projection of (x — y) onto Nx (y)- Then v G Nx (y), and so there exists 
yi — i> y, with yi E W (1 X, and Vi ^ v such that Vi G Nx [Vi)- So 

c?(x-i/,Arx(y)) = d{x-y,R+iv)) 

= lim d{x - y,R+{vi)) 

j— »oo 

= lim d{x - yi,R+{vi)) 

j— >oo 

> lim supd (x - yi,Nx {Vi 

rf(x-y,Arx(l/)) , d(x-yi,Nxiyi)) 

=^ n ^ limsup — ^ — — —. 

If -yll i^oo lF-2/i|| 

Thus equation [7 .41 holds . Therefore 

d(x-y, Nx (y)) 

lim inf — — — — > 1 implies limsuplip^p^-j,!! {x \ y) < I, 

y \\x y\\ y — >x 

so we may now consider only regular normal cones. 
By Lemma 1731 we deduce the following: 

d{x-y,Nx{y)^ + d (x - y, Nx (y)*^ = ||^ - ?/||^ for ?/ G X 



LIPSCHITZ BEHAVIOR OF THE ROBUST REGULARIZATION 26 

Since Tx (y)* = Nx (y) always [23, Theorem 6.28(a)], we apply Lemma 
I7.3l and get 

d(x-y, Nx (y)) +d{x-y,Tx {yT*f = \\x - yf for y e X. 

As Tx (y) C Tx (y)** Il23l Corollary 6.21], this implies that 

(7.5) d(^x-y,Nx{y)y + d{x-y,Tx{y)f > \\x - yf foryeX. 

Note that if X is nearly radial at x, then ■pz^'^ (x — y, Tx (y)) — * as 
e = ||x — yll I0,y e X. This means that 

l/lipl>lls_yll {x\y)> ^ d fx - y,Nx {y)) 1, 

so 

limsup lip<l||j;_y|| {x \y) < 1, 

where y — > x means y G X and y ^ x. 

Recall that $e has closed graph, and hence it is outer semicontinuous [|23l 
Theorem 5.7(a)]. It is also locally bounded, so 

lip <le (x) = max lip {x \ y) 

yeS,{x) 

by [|20l Theorem 1.42]. This gives us lim sup^__^o (^) ^ 1, or X is 
1 -peaceful at needed. 

If we assume that X is regular in a neighborhood of x, then Formula (17.51 ) 
is an equation. Furthermore, (17.11 ), (17.21) and (17.31) are all equations. Thus if 

lime_>olip (x) = 1, then 

||- ^ (x - y, Nx (y)) = 1/lip ^\\x-y\\ {x \ y) 1 asy ^ X, y X. 

and we have p^*^ (x — y,Tx (y)) as y — > x and y ^ x, which 
means that X is nearly radial at X. □ 

Finally, 1— peaceful sets are interesting in robust regularization for an- 
other reason. The Lipschitz modulus of the robust regularization over 1- 
peaceful sets have Lipschitz modulus bounded above by that of the original 
function, as the following result shows. 

Proposition 7.5. IfX is 1-peaceful and F : X -^MT" is locally Lipschitz at 
X, then 

lim sup lip (x) < lip F (x) . 

£^0 
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Proof. We use a set- valued chain rule [[231 Exercise 10.39]. Recall the for- 
mula F^= i^F o \x. The mapping (a;, u) ^ {x)nF^^ (n) is locally 

bounded because the map x ^ (x) is locally bounded. Thus 

lip (x) < lip <l>e {x) ■ max lip F (x) . 

a;el>e(x) 

By Theorem 17. 41 lim^^o lip {^) ^ 1- Also, since lip F : M" ^ M_|_ is up- 
per semicontinuous, limsup^_^Q max^gl.^^-^ lip F (x) <lipF(x). Taking 
limits to both sides gives us what we need. □ 

8. Nearly radial sets 

As highlighted in Section |71 nearly radial sets are 1-peaceful. In this 
section, we study the properties of nearly radial sets and give examples of 
nearly radial sets to illustrate their abundance in analysis. 

We contrast the definition of nearly radial sets given before Proposition 
14.51 with a stronger property introduced by [25], which is the uniform ver- 
sion of the same idea. This idea was called o{l)-convexity in [|25l. 

Definition 8.1. (nearly convex sets) A set X C M" is nearly convex at a 
point X G X if 

dist (y, X + Tx (x)) = o (||x — yll) as X, y — > X in X 

The set X is nearly convex if it is nearly convex at every point X. o 

Clearly if a set is nearly convex at a point, then it is nearly radial there, 
but the class of nearly radial sets is considerably broader. For example, the 
set 

X = {x G : X1X2 = 0} 

is nearly radial at the origin but not nearly convex there, since as — > 00 
the points x„ = (n"^, 0) and y„ = (0, n^^) approach the origin in X and 
yet 

dist(?/„, x„ + Tx{xn)) = 11'^ ^ o(||x„ - yn\\). 

It is immediate that convex sets are nearly convex, and hence nearly 
radial. A straightforward exercise shows that smooth manifolds are also 
nearly convex, and hence again nearly radial. These observations are both 
special cases of the following result, rather analogous to [|25l Theorem 2.2]. 
A set X C R" is amenable [23, Section lOF] at a point x G X if there is 
an open neighborhood \^ of x, a mapping F : V ^ M"*, and a closed 
convex set D C M™, such that 

xnv = {x eV : F{x) e D} 
(8.6) and X^ (F (x)) n X (VF (x)*) = {0} , 
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where Njj^-) denotes the normal cone to D, and A^(-) denotes null space. If 
in fact F is then we call X strongly amenable [|23l Definition 10.23] at 

X. 

Theorem 8.2. (amenable implies nearly radial) Suppose the set X C M" is 
amenable at the point x E X. Then X is nearly convex (and hence nearly 
radial) at x. 

Proof. Since X is amenable at x, we can suppose property (18.61) holds. Sup- 
pose without loss of generality x = 0, and consider a sequences of points 
2^r, ?/r — in the set X fl We want to show 

d\^i{yr,Xr + Tx{Xr)) = o(^\\Xr - VrW) ■ 

Without loss of generality we can suppose Xj- ^ Ur for all r, and denote the 
unit vectors \\xr — yr\\^^{xr — Ur) by Zr. We want to prove 

dr = min{||w + Zr\\ : w G Tx{xr)} — > 0. 

The unique minimizer Wr E Tx{xj.) in the above projection problem satis- 
fies 

di^- — 1 1 U] Y' I 1 1 
Wr + Zr e -Nx{Xr) = -V F (Xr)* Nd{F (^Xr)) 
{Wr,Wr + Zr) = 0, 

by ll23l Exercise 10.26(d)]. Choose vectors Ur E —Nr){F{xr)) such that 

Wr + Zr = V F{Xr)*Ur. 

We next observe that the sequence of vectors {ur} is bounded. Other- 
wise, we could choose a subsequence {ur'} satisfying oo, and 
then any limit point of the sequence of unit vectors must lie 
in the set -iVz)(F(0)) n X(VF(0)*), contradicting property (l8^ . 

We now have 

Q < dl = {Zr,VF{XryUr) = {V F {x r) Zr , Ur) 

= {S/F{Xr)Zr - \Xr - yr\'\F{Xr) - F{yr%Ur) 
+ - yr\~\F{Xr) - F{yr)],Ur) ■ 

The first term converges to zero, using the smoothness of the mapping F and 
the boundedness of the sequence {ur}. On the other hand, since the set D is 
convex, we have — E Tj:,{F(xr)), and Ur E —N£)(F{xr)) by 

assumption, so the second term is nonpositive, and the result follows. □ 

It is worth comparing these notions to a property that is slightly stronger 
still: prox-regularity (in the terminology of [i23i Section 13F]), or 0(2)- 
convexity [|25l . 
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Definition 8.3. (prox-regular sets) A set X C M" is prox-regular at a point 

dist (y, X + Tx (x)) = O {\\x — as x, y ^ X in X.o 

Theorem |8.2| (amenable implies nearly radial) is analogous to the fact that 
strong amenability implies prox-regularity [,23. Proposition 13.32] (and also 
to (251 Proposition 2.3]). 

The class of nearly radial sets is very broad, as the following easy result 
(which fails for nearly convex sets) emphasizes. 

Proposition 8.4. ( unions) If the sets Xi, X2, . . . , Xn are each nearly radial 
at the point x G ^jXj, then so is the union U^Xj. 

Proof. If the result fails, there is a sequence of points Xr ^ x'm U^X^ and 
real e > such that 



(8.7) dist ( , Tyj^x, {xr))>e for all r. 

\||a; — Xrll / 

By taking a subsequence, we can suppose that there is an index i such that 
Xr G Xj for all r. But then we know 

dist(^^^,Tx,(x,)) ^0, 

which contradicts inequality (|8.7I) . since Tx^ {xr) C Tyj^x, (a^r). D 

A key concept in variational analysis is the idea of Clarke regularity (see 
for example [8l |9l [231). We make no essential use of this concept in our 
development, but it is worth remarking on the relationship (or lack of it) 
between the nearly radial property and Clarke regularity. Note first that 
nearly radial sets need not be Clarke regular: the union of the two coordinate 
axes in is nearly radial at the origin, for example, but it is not Clarke 
regular there. 

On the other hand, Clarke regular sets need not be nearly radial. 



Example 8.5. Consider the function / : M ^ M defined by 

/(^) = 



2-„ _ 2-n-i(2 - 2'^+i|x|)i+2 " if 2-"-i < |x| < 2-" (n G N) 
ifa; = 0. 

The function / is even, and its graph consists of concave segments on each 
interval x G [2~"~^, 2~"], passing through the point 2^"(1, 1) with left de- 
rivative zero, and through the point 2^"^^ (1, 1) with right derivative 1+2^". 
A routine calculation now shows that this function is everywhere regular, 
and hence its epigraph epi / is everywhere Clarke regular. However, epi / 
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is not nearly radial at the origin. To see this, observe that for each n G N, if 
we consider the sequence x„ = 2~"(1, 1) — > (0, 0), then we have 

Tepi/(xO = :y > (l + 2i-")max{x,0}}, 



so 



dist(0,a;„ + Tepi/(x„)) 



n/2 ' 



contradicting the definition of a nearly radial set. o 

This is yet another attractive property for semi-algebraic sets. 

Theorem 8.6. (semi-algebraic sets) Semi-algebraic sets are nearly radial. 

Proof. Suppose the origin lies in a semi-algebraic set X C M". We will 
show that X is nearly radial at the origin. 
If the result fails, then there is a real 5 > and a sequence of points 



Vr 



in X such that 





Vr 








WVrW 



> 6 for all M G TxiVr)- 



Hence for each index r there exists a real 7,. > such that 

Z-Vr , Vr 



.\Z-yr\\ IWr.. 

Consequently, each point yr lies in the set 



> 6 for all 2; G X such that < — yr\\ < 7r- 



X 







2/ G X I 37 > so 



y ^ y 



\\y\\ 

V2 G X \ {y} with \\z 



> 6 

y\\ < 7 



so G cIXq. 

By quantifier elimination (see for example the discussion of the Tarski- 
Seidenberg Theorem in [21 p. 62]), the set Xq is semi- algebraic. Hence the 
Curve Selection Lemma (see fD, p. 98] and [fT9ll ) shows that there is a real- 
analytic path p : [0, 1] such that p(0) = and p{t) G Xq for all 
t E (0, 1]. For some positive integer k and nonzero vector g E we have, 
for small t > 0, 

pit) = gt^ + 0{t^+^) 
p\t) = kgt'-' + Oit'), 
and in particular both p(t) and p'{t) are nonzero. For any such t we know 



z - p{t) p{t) 



+ 



k-pwii urn 



> 6 



LIPSCHITZ BEHAVIOR OF THE ROBUST REGULARIZATION 



31 



for any point z E X \ {pit)} close to p{t). Hence for any real s ^ t close 
to t we have 

p{s)-p{t) , p{t) 



Ms) -Pirn imw 

Taking the limit as s t t shows 

Pjt) p'{t) 

Mm M 

for all small t > 0. But since 



> 5. 



> 5 



mMm 



lim 



9_ 

\9\ 



lim 

m 



P'it) 



\p 



this is a contradiction. 



□ 



By contrast, semi- algebraic sets need not be nearly convex. For example, 
the union of the two coordinate axes in is semi-algebraic, but it is not 
nearly convex at the origin. 
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